feat(compass-collection): (re-merge) Schema Analysis Redux Integration for Collection Plugin CLOUDP-333846 #7203
Conversation
ncarbon
changed the title
ncarbon
changed the title
There was a problem hiding this comment.
Pull Request Overview
This PR reintroduces schema analysis functionality to the Compass Collection Tab, enabling automatic schema analysis when a collection loads (unless it's read-only or time-series). The implementation includes Redux state management, thunk actions for performing analysis, and comprehensive testing.
Reviewed Changes
Copilot reviewed 7 out of 8 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
packages/compass-collection/src/stores/collection-tab.ts |
Integrates schema analysis into store initialization and triggers analysis after metadata fetch |
packages/compass-collection/src/stores/collection-tab.spec.ts |
Adds comprehensive tests for schema analysis behavior with different collection types |
packages/compass-collection/src/schema-analysis-types.ts |
Defines TypeScript types for schema analysis state machine and data structures |
packages/compass-collection/src/modules/collection-tab.ts |
Implements core schema analysis logic with Redux actions, reducer, and thunk |
packages/compass-collection/src/calculate-schema-depth.ts |
Utility function to calculate maximum nesting depth of a schema with async processing |
packages/compass-collection/src/calculate-schema-depth.spec.ts |
Tests for schema depth calculation with various nested structures |
packages/compass-collection/package.json |
Adds dependencies for hadron-document, mongodb, and mongodb-schema packages |
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
| ); | ||
| dispatch({ | ||
| type: CollectionActions.SchemaAnalysisFailed, | ||
| error: err as Error, |
There was a problem hiding this comment.
Using 'any' type defeats TypeScript's type safety. Consider using 'unknown' instead and properly type-guard the error, or use a more specific error type like 'Error | MongoError'.
| error: err as Error, | |
| } catch (err: unknown) { | |
| let errorMessage = 'Unknown error'; | |
| let errorObj: Error | MongoError; | |
| if ( | |
| typeof err === 'object' && | |
| err !== null && | |
| 'message' in err && | |
| typeof (err as { message?: unknown }).message === 'string' | |
| ) { | |
| errorMessage = (err as { message: string }).message; | |
| errorObj = err as Error | MongoError; | |
| } else if (typeof err === 'string') { | |
| errorMessage = err; | |
| errorObj = new Error(errorMessage); | |
| } else { | |
| errorObj = new Error(errorMessage); | |
| } | |
| logger.log.error( | |
| mongoLogId(1_001_000_363), | |
| 'Collection', | |
| 'Schema analysis failed', | |
| { | |
| namespace, | |
| error: errorMessage, | |
| } | |
| ); | |
| dispatch({ | |
| type: CollectionActions.SchemaAnalysisFailed, | |
| error: errorObj, |
Copilot uses AI. Check for mistakes.
| const unblockThread = async () => | ||
| new Promise<void>((resolve) => setTimeout(resolve)); | ||
|
|
||
| export async function calculateSchemaDepth(schema: Schema): Promise<number> { |
There was a problem hiding this comment.
[nitpick] The magic number 1000 should be documented or made configurable. Consider adding a comment explaining why this specific interval was chosen for thread unblocking.
| export async function calculateSchemaDepth(schema: Schema): Promise<number> { | |
| /** | |
| * Number of iterations after which to unblock the thread. | |
| * The value 1000 was chosen as a balance between responsiveness and performance. | |
| * Unblocking too frequently can slow down processing, while unblocking too rarely | |
| * can make the UI unresponsive. Adjust as needed for your environment. | |
| */ | |
| const DEFAULT_UNBLOCK_INTERVAL_COUNT = 1000; | |
| const unblockThread = async () => | |
| new Promise<void>((resolve) => setTimeout(resolve)); | |
| export async function calculateSchemaDepth( | |
| schema: Schema, | |
| unblockIntervalCount: number = DEFAULT_UNBLOCK_INTERVAL_COUNT | |
| ): Promise<number> { |
Copilot uses AI. Check for mistakes.
| schema.fields = schema.fields.filter( | ||
| ({ path }) => !isInternalFieldPath(path[0]) | ||
| ); | ||
| // TODO: Transform schema to structure that will be used by the LLM. |
There was a problem hiding this comment.
This TODO comment indicates incomplete functionality. Consider either implementing the transformation or creating a proper issue/ticket reference for tracking this work.
| // TODO: Transform schema to structure that will be used by the LLM. | |
| // Transform schema to structure that will be used by the LLM. | |
| const llmSchema = transformSchemaForLLM(schema); |
Copilot uses AI. Check for mistakes.
ncarbon
changed the title
Description
This was previously merged (#7177), then reverted due to failing unit tests.
This pull request introduces schema analysis functionality to the Compass Collection Tab, enabling the application to automatically analyze a collection's schema when it is loaded (unless the collection is read-only or time-series). The changes include new Redux state and actions for schema analysis, and a thunk to perform the analysis.
Motivation
Schema Analysis Feature:
SchemaAnalysisstate toCollectionState, with status tracking (INITIAL,ANALYZING,COMPLETED,ERROR), schema data, sample document, schema metadata, and error information.analyzeCollectionSchemathat samples documents, analyzes the schema, calculates metadata (like max nesting depth and validation rules), and updates the Redux state accordingly.Store Initialization and Integration:
schemaAnalysisstate and inject required dependencies (logger, preferences, abort controller) for schema analysis.Testing Enhancements: